Concentration inequalities using the entropy method
نویسندگان
چکیده
We investigate a new methodology, worked out by Ledoux and Mas-sart, to prove concentration-of-measure inequalities. The method is based on certain modified logarithmic Sobolev inequalities. We provide some very simple and general ready-to-use inequalities. One of these inequalities may be considered as an exponential version of the Efron-Stein inequality. The main purpose of this paper is to point out the simplicity and the generality of the approach. We show how the new method can recover many of Talagrand's revolutionary inequalities and provide new applications in a variety of problems including Rademacher averages, Rademacher chaos, the number of certain small subgraphs in a random graph, and the minimum of the empirical risk in some statistical estimation problems.
منابع مشابه
Entropy-Based Concentration Inequalities for Dependent Variables
We provide new concentration inequalities for functions of dependent variables. The work extends that of Janson (2004), which proposes concentration inequalities using a combination of the Laplace transform and the idea of fractional graph coloring, as well as many works that derive concentration inequalities using the entropy method (see, e.g., (Boucheron et al., 2003)). We give inequalities f...
متن کاملSubadditivity of Matrix Φ-entropy and Concentration of Random Matrices
Matrix concentration inequalities provide a direct way to bound the typical spectral norm of a random matrix. The methods for establishing these results often parallel classical arguments, such as the Laplace transform method. This work develops a matrix extension of the entropy method, and it applies these ideas to obtain some matrix concentration inequalities.
متن کاملSubadditivity of Matrix phi-Entropy and Concentration of Random Matrices
Matrix concentration inequalities provide a direct way to bound the typical spectral norm of a random matrix. The methods for establishing these results often parallel classical arguments, such as the Laplace transform method. This work develops a matrix extension of the entropy method, and it applies these ideas to obtain some matrix concentration inequalities.
متن کاملConcentration inequalities for functions of independent variables
Following the entropy method this paper presents general concentration inequalities, which can be applied to combinatorial optimization and empirical processes. The inequalities give improved concentration results for optimal travelling salesmen tours, Steiner trees and the eigenvalues of random symmetric matrices. 1 Introduction Since its appearance in 1995 Talagrands convex distance inequali...
متن کاملConcentration of Measure Inequalities in Information Theory, Communications, and Coding Concentration of Measure Inequalities in Information Theory, Communications, and Coding
During the last two decades, concentration inequalities have been the subject of exciting developments in various areas, including convex geometry, functional analysis, statistical physics, high-dimensional statistics, pure and applied probability theory (e.g., concentration of measure phenomena in random graphs, random matrices, and percolation), information theory, theoretical computer scienc...
متن کاملConcentration of Measure Inequalities in Information Theory, Communications and Coding
Concentration inequalities have been the subject of exciting developments during the last two decades, and they have been intensively studied and used as a powerful tool in various areas. These include convex geometry, functional analysis, statistical physics, statistics, pure and applied probability theory (e.g., concentration of measure phenomena in random graphs, random matrices and percolat...
متن کامل